RESUMEN
Objectives: Using a historical control or external control arm (ECA) to augment or replace a concurrent control arm in a randomized trial is a hot topic given the challenge of patient recruitment in rare diseases or during COVID-19 pandemic. The FDA released draft guidance in 2021 on effectiveness and safety submissions using real-world evidence. While the guidance focuses mainly on elements of study design and data source selection, there is a lack of consensus in the selection of appropriate statistical methods when constructing an ECA. This study discusses rigorous statistical methodology for ECA-supported trials in regulatory or HTA submissions. Method(s): Targeted literature reviews of statistical simulations comparing methods for ECA in statistical journals were performed. The articles compared commonly used ECA-construction and analysis methods were selected and summarized, including but not limited to propensity score (PS)-based matching, weighting, and stratification, and PS plus Bayesian integrated approaches. Result(s): Type I error, power, bias, and coverage probability are common criteria used to compare different methods. When imbalances only exist in known baseline covariates and the outcome distributions are the same between the trial concurrent control and ECA, the PS method alone or paired with commensurate prior yield almost unbiased estimates, good Type I errors, and coverage probability. PS plus Bayesian approaches have wider interval width and lower power compared with PS-only methods. When there is a change in the outcome distribution over time, the PS (matching or IPTW) and commensurate prior integrated methods yield the smallest biases among all methods. Conclusion(s): PS and Bayesian integrated methods outperformed the PS-only methods in terms of bias and Type I error when outcome distribution changed with current trial control. A "sweet spot" that balances all criteria through trial-specific simulations could provide the ideal setting of trial analyses plan based on specific trial design and scenarios.Copyright © 2023
RESUMEN
Objectives: Using a historical control or external control arm (ECA) to augment or replace a concurrent control arm in a randomized trial is a hot topic given the challenge of patient recruitment in rare diseases or during COVID-19 pandemic. FDA released several draft guidance in 2021 on effectiveness and safety submissions using real-world evidence. While the guidance focuses mainly on elements of study design and data source selection, there is a lack of consensus in the selection of appropriate statistical methods when constructing an ECA. This study aims to discuss rigorous statistical methodology for ECA-supported trial in regulatory or HTA submissions. Method(s): Targeted literature reviews of statistical simulations comparing methods for ECA in statistical journals were performed. The articles compare commonly used ECA-construction and analysis methods were selected and summarized, including but not limited to propensity-score (PS) based- matching, weighting, stratification, and PS plus Bayesian integrated approaches. Result(s): Type I error, power, bias, and coverage probability are common criteria used to compare different methods. When imbalances only exist in known baseline covariates and the outcome distribution are the same between the trial concurrent control and ECA, PS method alone or paired with commensurate prior yield almost unbiased estimates, good Type I errors, and coverage probability. PS plus Bayesian approaches have wider interval width and lower power compared with PS only methods. When there is a change in the outcome distribution over time, PS (matching or IPTW) and commensurate prior integrated method yield smallest biases among all methods. Conclusion(s): PS and Bayesian integrated methods outperformed the PS only methods in terms of bias and type I error when outcome distribution changed with current trial control. A "sweet spot" that balances all criteria through trial-specific simulations could provide the ideal setting of trial analyses plan based on specific trial design and scenarios. Copyright © 2022
RESUMEN
Background: During the COVID-19 pandemic, many aspects of traditional clinical trials have been affected worldwide. Recruitment of patients and on-site visits have been challenging when not compromised. Many groups have turned to the possibility of replacing their randomized standard of care (SOC) arm with a real-world (RW) external control arm (ECA). Unlike randomized controlled trials (RCTs) that have international guidelines, the use of an ECA is not subject to any consensus. Objectives: The aim of this study is to guide the design of an ECA (when it is justified or recommended) from different context and data sources. Methods: We propose to summarize the evidence into a decisional matrix. We crossed popular data sources (RW data collected prospectively, RW data obtained from retrospective chart review, administrative or insurance data, and clinical trial data) with situations where the use of an ECA could be justified or beneficial (regulatory submission, health economics and outcomes research [HEOR] investigation, hypothesis generation). Our reflection was influenced by our consulting practice at Evidera and by United States Food and Drug Administration (FDA) guidelines on the use of RW data. We developed a framework that should help researchers to build a quality ECA. Building an ECA should be based on a clear research question that will inform: (1) design and data source(s) to be used;(2) selection of control that will limit biases;and (3) adjustment methods that allow fair comparison with the treated arm. Results: Good ECAs would have a clear and accepted SOC treatment (limited changes in medical practice), standardized variables and definitions, similar outcome evaluations, and validity of the variables. Conclusions: When it comes to ECA, there is no one size fits all solution. The ECA should not replace RCTs but be considered as a complement tool to provide additional evidence to medical research.